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Abstract —  Now-a-days  Science  and  Technology  have  made  the  human  world  so  easy  but  still  some  physically 
and  visually  challenged  people  suffer  from  communication  with  others.  In  this  project,  we  are  going  to 
propose  a  new  system  prototype  called  communication  among  Blind,  deaf  and  dumb  people  .This  will  helps  the 
disabled  people  to  overcome  their  difficulties  in  communicating  with  some  other  people  with  disabilities  or 
normal  people.  The  blind  people  will  communicate  through  the  speakers,  the  deaf  and  dumb  people  will  see 
through  it  and  reply  through  typing  in  a  terminal  .These  are  all  done  as  an  application  ,  so  that  will  be  easily 
understand  by  the  people  with  disabilities. 
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I.  INTRODUCTION 

Social  cognition  is  the  product  of  the  behavioural  and 
cognitive  processes  dedicated  to  build  our  social  world.  In 
humans  as  well  as  in  other  species,  many  social  cues  take 
the  form  of  body  language  and  therefore  require  vision  in 
order  to  be  understood  [1],  For  example  the  imitative 
behaviour  in  newborn  infants  provides  an  innate 
foundation  for  understanding  inner  states  belonging  to 
other  persons  and  for  mapping  others'  actions  into  actions 
of  their  own  body  [2],  Since  vision  directly  enables  the 
contextual  perception  of  surrounding  objects,  the  absence 
of  vision  adversely  affects  not  only  language,  motor,  and 
cognitive  development  [3],  but  also  social  competence  [4- 
6]  in  visually  impaired  people.  For  example,  since  totally 
blind  children  lack  visual  references,  they  might  not  be 
aware  of  social  cues  in  the  form  of  body  gestures  while 
communicating  with  others.  Similarly,  low  vision  children 
might  not  correctly  express  and  interpret  body  language 
because  their  imitative  behaviours  during  infancy  was 
prevented  due  to  their  degraded  vision.  As  a  consequence, 
visually  impaired  children  often  face  problems  in 
integrating  socially.  Preschool-age  children  with  visual 
disabilities  often  have  difficulty  engaging  in  positive  social 
interactions.  Many  do  not  display  a  full  range  of  play 
behaviours  [4-11],  spend  more  time  in  solitary  play 
interacting  more  with  adults  than  with  their  sighted  peers 
[6,  9,  10,  1217],  and  are  typically  unable  to  demonstrate 
peer-related  social  competence  [18-20].  This  social 


impairment  gives  rise  to  feelings  of  frustration  and  fear  of 
interaction,  rather  than  self  efficacy  and  independence, 
which  characterize  the  social  experience  of  typical 
children. 

II.  RELATED  WORK 

1.  Participants 

Six  visually  impaired  (3  females,  age  range:  mean  age:  42 
years  old)  and  eight  sighted  (4  females,  age  range:  mean 
age:  40  years  old)  adults  participated  in  the  study. 
Furthermore,  seventeen  visually  impaired  (12  females, 
mean  age:  12  years  old)  and  twelve  sighted  (4  females, 
mean  age:  13  years  old)  children  aged  6  to  17  participated 
to  the  study.  Adults  and  children  groups  were  aged- 
matched  (t(12)  =  -0.15;  P  =  0.88;  t(27)  =  0.35;  P  =  0.72). 

2.  ABBI 

ABBI  is  a  wearable  electronic  device  positioned  on  the 
wrist  that  produces  sounds  when  moved.  It  comprises  an 
ABBI  bracelet  and  a  mobile  phone  with  an  ad-hoc 
application  that  allows  one  to  control  the  sound  and  the 
volume.  The  bracelet  and  the  mobile  phone  communicate 
via  Bluetooth  Low-Energy  [23,  28]. 

Audio  Social  Interaction  Task  and  Protocol 

The  task  was  performed  in  a  dark  room  for  better  tracking 
movements  with  the  motion  captive  system.  The  task 
required  the  experimenter  to  wear  ABBI  on  the  wrist  and 
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move  the  arm  making  three  different  shapes  (straight  line, 
circle  and  lemniscates  symbol)  in  random  order.  The 
participant  stood  in  front  of  the  experimenter  and  had  to 
follow  the  experimenter  movement  that  the  participant 
perceived  from  the  moving  sound  source  (i.e.  the  ABBI 
bracelet).  Each  trial  lasted  30  seconds,  and  each  shape  was 
performed  5  times  in  a  random  order,  for  a  total  of  15-20 
minutes  per  participant  including  breaks.  Blind  and 
blindfolded  sighted  participants  performed  15  trials  in 
which  the  shape  produced  by  the  experimenter  was  known 
in  advance  and  6  trials  in  which  it  was  not 

III.  LITERATURE  SURVEY 

1.  Multi-modal  Interfaces  for  Interaction- 
Communication  between  Hearing  and  Visually 
Impaired  Individuals:  Problems  &  Issues 

In  this  paper  the  author  said  one  important  and  challenging 
problem  in  human  interaction  is  the  communication 
between  blind  and  deaf  individuals.  The  challenge  here 
involves  several  cases:  (i)  first  case  is  a  deaf  person  usually 
does  not  speak  in  order  a  blind  person  to  hear  him/her;  (ii) 
second  case  is  when  a  blind  person  speaks  a  deaf  person 
cannot  hear;  (iii)  third  case  is  when  a  deaf  person  makes 
sign  language  signs  a  blind  person  cannot  see  them.  Thus, 
this  paper  presents  a  study  on  multi-modal  interfaces, 
issues  and  problems  for  establishing  communication  and 
interaction  between  blind  and  deaf  persons.  A  system- 
prototype  Tyflos-  Koufos  is  proposed  in  an  effort  for 
offering  solutions  to  these  challenges. 

2.  Design  of  an  Assistive  Communication  Glove  using 
Combined  Sensory  Channels 

This  paper  presents  a  new  design  of  a  wireless  sensor  glove 
developed  for  American  Sign  Language  Finger  spelling 
gesture  recognition.  Five  contact  sensors  are  installed  on 
the  glove,  in  addition  to  five  flex  sensors  on  the  fingers  and 
a  3D  accelerometer  on  the  back  of  the  hand.  Each  pair  of 
flex  and  contact  sensors  are  combined  into  the  same  input 
channel  on  the  BSN  node  in  order  to  save  the  number  of 
channels  and  the  installation  area.  After  which,  the  signal  is 
analyzed  and  separated  back  into  flex  and  contact  features 
by  software.  With  electrical  contacts  and  wirings  made  of 
conductive  fabric  and  threads,  the  glove  design  has  become 
thinner  and  more  flexible.  For  validation,  ASL  finger 
spelling  gesture  recognition  experiments  have  been 
performed  on  signals  collected  from  six  speech-impaired 
subjects  and  a  normal  subject.  With  the  new  sensor  glove 
design,  the  experimental  results  have  shown  a  significant 
increase  in  classification  accuracy. 
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3.  Automation  of  the  Arabic  Sign  Language 
Recognition 

Interfaces  in  sign  language  systems  can  be  categorized  as 
direct-device  or  vision-based.  The  direct-device  approach 
uses  measurement  devices  that  are  in  direct  contact  with 
the  hand  such  as  instrumented  gloves,  flexion  sensors,  styli 
and  position-tracking  devices.  On  the  other  hand,  the 
vision-based  approach  captures  the  movement  of  the 
singer's  hand  using  a  camera  that  is  sometimes  aided  by 
making  the  signer  wear  a  glove  that  has  painted  areas 
indicating  the  positions  of  the  fingers  or  knuckles.  The 
main  advantage  of  vision-based  systems  is  that  the  user 
isn't  encumbered  by  any  complex  devices;  their  main 
disadvantage,  however,  is  that  they  require  a  large  amount 
of  computation  just  to  extract  the  hand  position  before 
performing  any  analysis  on  the  images.  In  this  paper  we 
will  discuss  only  the  directed -devise  methods. 

4.  Design  of  new  digital  blind  feed  forward  timing 
recovery  algorithm  for  linear  modulations  in  digital 
communication  receivers 

Synchronizers  are  a  significant  part  of  digital 
communication  receivers  since  they  are  used  to  estimate 
the  received  signal  timing.  Among  these  synchronizers, 
non-data-aided  (NDA)  or  blind  synchronizers  are  even 
more  important  due  to  bandwidth  efficiency.  In  this  study, 
a  different  approach  is  investigated  to  derive  new  feed 
forward  NDA  timing  estimators  based  on  Newton 
algorithm.  Two  novel  estimators  with  four  and  two 
samples  per  symbol  are  presented  and  compared  with 
traditional  timing  estimators,  respectively.  One  of  the  main 
advantages  of  the  second  proposed  estimator  is  the  ability 
to  control  the  loop  gain  automatically.  This  ability 
improves  the  convergence  speed  and  robustness  of  the 
design.  Simulation  results  show  that  the  presented 
estimators  outperform  the  conventional  estimators  over  a 
noisy  channel,  especially  at  small  excess  bandwidth  and 
low  signal  to  noise  ratio. 

5.  ACE  Assisted  Communication  for  Education: 
Architecture  to  support  Blind  &  Deaf 
communication 

Deaf  and  blind  students  face  communication  barriers  that 
are  constantly  present  in  their  daily  lives.  These  barriers 
arise  naturally  since  the  deaf  community;  the  blind 
community  and  the  rest  of  the  students  and  teachers  use 
different  languages  and  different  channels  to  communicate. 
These  barriers  have  a  significant  impact  in  the  academic, 
personal  and  professional  development  of  deaf  and  blind 
students.  Using  automatic  tools  to  assist  the  fluid 
communication  between  people  using  different  languages 
and  different  channels  of  communication  might 
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significantly  promote  the  social  inclusion  of  disabled 
students.  In  this  paper  we  describe  the  ACE  architecture 
which  incorporates  Virtual  Sign,  a  translator  for  sign 
language  and  other  components  to  allow  for  the  real  time 
translation  between  sign  and  oral  languages.  This 
architecture  supports  any  sign  language.  We  expect  ACE 
will  provide  ways  for  the  fluid  communication  among  deaf 
people,  blind  people  and  those  not  constrained  by  these 
disabilities. 


IV.  EXISTING  SYSTEM 

In  existing  system  Sign  Language  is  the  only  way  of 
communication  for  speech  impaired  people.  A  the  general 
people  unable  to  understand  the  sign  language  ,  so  it 
becomes  very  difficult  for  a  speech  impaired  person  to 
communicate  with  them.  There  is  no  system  to 
communicate  speech  impaired  people. 


V.  PROPOSED  SYSTEM 

In  this  project  an  electronic  speaking  system  and  a  terminal 
for  typing  was  developed  to  ease  the  communication 
process  of  speech  impaired  people  and  the  speaker  is 
developed  for  speaking  for  the  visually  impaired  people 
.This  makes  the  disabled  people  to  communicate  easily 
without  the  help  of  others. 


VI.  MODULES 

1.  BLIND  MODULE 

2.  DEAF  MODULE 

3.  DUMB  MODULE 

1.  BLIND  MODULE 

In  older  days,  they  use  Braille  system  to  communicate  with 
blind  people,  to  overcome  this,  we  use  voice  controller 
system  to  communicate  with  the  blind  people  .In  this,  the 
blind  people  can  talk  through  the  voice  controller  to 
communicate  with  other  people.  In  another  case,  we  have 
an  app  like,  the  blind  people  can  call  or  message  to  the 
respective  person  in  their  contact  by  drawing  their  own 
gestures  respective  of  their  contacts. 

It  is  much  more  useful  for  the  blind  people  to 
communicate. 

2.  DEAF  MODULE 

In  this  deaf  module,  we  can  able  to  communicate  with  the 
deaf  people,  using  the  terminal.  They  can  type  on  whatever 
they  want  to  say  and  it  will  be  displayed  in  the  terminal 
page  of  other  people 


3.  DUMB  MODULE 

Sign  languages  are  languages  that  are  used  by  dumb  people 
to  express  their  feeling  through  manual  articulations  in 
combination  with  non-manual  elements.  Sign  languages 
are  also  natural  languages  in  their  own  grammar  and 
lexicon.12  Sign  languages  are  not  universal  as  they  are 
not  mutually  intelligible,121  although  there  are  also  striking 
similarities  among  sign  languages. 

Linguists  consider  spoken  and  signed  communication  to  be 
types  of  natural  language  as  they  both  express  their  own 
thoughts  in  their  way  of  communication.  Sign  language 
should  not  be  confused  with  body  language,  as  they  are  a 
type  of  nonverbal  communication. 


VII.  FIGURES  AND  TABLES 

VOICE  CONTROL 


Connected! 


Fig.  1  Blind  module 


TERMINAL 


Q^nter  command  here 


Fig  .  2  Dumb  module 


VIII.  FUTURE  ENHANCEMENT 

Future  work  would  utilize  a  cyber  glove  and  more 
sophisticated  featured  extraction  algorithm  will  be  utilized. 
Upon  successful  implementation  of  the  proposed  system, 
another  system  would  be  developed  to  transfer  spoken 
language  into  signs  using  handheld  devices  like  PDA. 
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IX.  CONCLUSION 

The  acquisition  of  spatial  hearing  is  typically  a  good 
indicator  of  the  future  ability  to  independently  navigate  in 
the  environment  and  engage  in  positive  social  interaction 
with  peers.  While  for  sighted  individuals  the  visual 
feedback  represents  the  most  important  incentive  for 
actions  and  thus  for  the  development  of  mobility  and  social 
skills  [32],  visually  impaired  individuals  strongly  rely  on 
auditory  landmarks  to  encode  spatial  and  social 
information.  Indeed,  it  has  been  shown  that  visually 
impaired  children  often  have  difficulties  engaging  in 
positive  social  interactions,  making  their  assimilation  into 
preschool  programs  difficult  and  giving  rise  to  counter¬ 
productive  feelings  of  frustration.  In  this  study,  we 
proposed  an  audio  social  interaction  task  in  which  two 
agents  have  to  interact  thanks  to  an  audio  source  in  order  to 
synchronize  their  arm  movements.  As  a  measure  of  social 
competence,  the  application  of  the  Granger  Causality 
method  allows  the  quantification  of  sensor  motor 
information  flow  between  partners.  This  is  the  first  attempt 
at  the  evaluation  of  joint  action  coordination  in  visually 
impaired  people.  More  importantly,  we  also  measured  the 
effects  of  an  extensive  but  entertaining  interventional 
program  with  ABBI  on  the  development  of  social 
competence  in  visually  impaired  children  aged  7-17  years 
old. 
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